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Abstract 


Information literacy instruction presents a difficult balance between quantity and quality, 
particularly for large-scale general education courses. This paper discusses the overhaul of 
the freshman composition instruction program at the University of Maryland Libraries, 
focusing on the transition from survey assessments to a student-centered and mixed- 
methods approach using qualitative reflections, rubrics, and the evaluation of student 
artifacts. The article discusses the progression from a pilot assessment program using 
Twitter as a data collection model to the implementation of a robust and multi-layered 
assessment using both qualitative feedback from students and the evaluation of student 
artifacts. Each assessment includes detailed collection methods and customized rubrics for 
evaluation of student responses. While information literacy assessment has been covered 
extensively in the literature, few articles discuss the use of qualitative student responses on a 
large scale (4,000 participants per year). The article also discusses the re-structuring of an 
assessment program around the ACRL Framework for Information Literacy, which is 
incorporated throughout the project from the pilot up through the full implementation of 
the final program. 
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Using the ACRL Framework to Develop a Student-Centered 
Model for Program-Level Assessment 


Introduction 

Large scale information literacy programming is a constant balance between quality and 
quantity, requiring partnership with hundreds of sections of a course, or courses, and 
enormous human, physical, and fiscal resources. As a result, information literacy instruction 
programs are often standardized—using the same teaching outline from section to section— 
and rely on newer, less experienced library instructors with heavy teaching loads to meet 
demand for instruction. At the University of Maryland (UMD) Libraries, this is best 
demonstrated in the first-year composition instruction program. Extending as far back as 
1995, this program has provided information literacy instruction to 95% of first year 
composition (ENGL101) sections, leading more than 200 sessions per year for nearly 4,000 
students. 

In fall of 2014, UMD Libraries' first-year composition instruction program was beginning 
to show its age. Lesson plans were based on a script, which had changed little in the 20-year 
history of the program, and assessment centered on a four-question survey that measured 
students' abilities to perform basic skills, such as identifying Boolean operators. In spring 
2015, librarians began the process of overhauling the program, starting with the learning 
outcomes and moving up through assessment. The intent was to shift the focus from a 
lecture-based format that emphasized search strategies, to an active learning curriculum 
intended to support the development of higher-level critical thinking skills. Informed by 
ACRL’s Framework for Information Literacy, the revised teaching outline introduces students 
to the threshold concepts through a student-centered lesson plan. New activities include a 
brainstorming process that guides students through crafting a research question and 
identifying keywords for database searching ( Research as Inquiry and Searching as Strategic 
Exploration), as well as an evaluating activity in which students examine sources for 
credibility ( Authority is Constructed and Contextual ) and suitability for inclusion in their class 
assignments ( Information Creation as a Process). Librarians also lead students through a small 
group discussion in which they identify a topic and analyze how seeking out multiple 
perspectives on that topic can strengthen an individual’s understanding ( Scholarship as 
Conversation). 
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Although updating the lesson plans was time-intensive, the most challenging aspect of the 
overhaul was not in the teaching, but the assessment. The typical solutions to programmatic 
assessment—pre- and post-tests, surveys, or the evaluation of small samples of student 
work—felt passive and distanced from learners, which did not match the critical and 
student-centered tone of our new instruction program. While student-centered assessment 
models, such as portfolio reviews or research journals, demonstrated the respect for 
individuality and the existence of multiple experiences we were seeking, they could not scale 
to the thousands of students involved in our program. We also faced the challenge of 
assessing learning in our new lesson plan, which was tied directly to ACRL’s Framework for 
Information Literacy. Although a wealth of literature has been published on teaching with the 
Framework, little practical assessment information is currently available. To address this gap, 
the researchers created an assessment strategy that combines the scalability of a survey with 
the intentionality of qualitative research. This article discusses the process through which 
our aging information literacy program was reinvigorated, focusing on the transition from a 
multiple-choice survey to an iterative, student-centered, and critically grounded assessment 
model mapped to ACRL's Framework for Information Literacy. 

Literature Review 

The literature is replete with studies assessing information literacy instruction at both the 
programmatic level and at the level of individual one-shot sessions. Researchers have 
employed assessment tools such as pre- and post-tests (Gilbert, 2009; Bryan & Karshmer, 
2013; Swoger, 2011); surveys and questionnaires designed to test students’ long-term 
retention of skills (Wong, Chan, & Chu, 2006); student reflections in the form of journals 
(Warner, 2003) and in-class activities such as one-minute papers (Choinski & Emanuel, 
2006) or a “start/stop” exercise (Flaspohler, 2003); and assessment of student artifacts 
(Holliday, et al., 2015; Diller & Phelps, 2008). Most of these studies describe relatively small- 
scale assessment projects. For instance, Warner’s (2003) pilot assessment involved only 48 
students, while Diller and Phelps’ study had about 200 student participants. And while other 
programs assessed artifacts from over 500 students (for example, Wong, Chan and Chu 
(2006) surveyed 688 users), none approached the scale of this project, which analyzes over 
1200 student responses per semester. This study aims to contribute to the literature by 
providing a student-centered model for large-scale information literacy instruction 
assessment that can be adopted by any institution facing the challenge of programmatic 
assessment. 
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This study also seeks to contribute to emerging discussions about how to connect 
assessment to the ACRL Framework for Information Literacy for Higher Education. Because 
ACRL’s transition from the Information Literacy Competency Standards for Higher Education to 
the Framework is still relatively recent, much of the literature on outcomes-based 
information literacy instruction is tied to the Information Literacy Competency Standards. 
While there is no dearth of scholarship examining the theoretical underpinnings and 
practical applications of the Framework for instruction, few articles have directly addressed 
the issue of designing assessment tools using the Framework. Oakleaf (2014) and Anderson 
(2015) have both offered guidance for instruction librarians making the transition to 
assessing using the Framework. Oakleaf addresses concerns about moving away from the 
Information Literacy Competency Standards, with its built-in learning outcomes, to the 
Framework’s threshold concepts. She offers a “roadmap” for translating the threshold 
concepts into student learning outcomes, which can be measured through assessment of 
appropriate student artifacts. Anderson also acknowledges that the move to the Framework 
represents a significant shift; she suggests that assessment should mirror the Framework’s 
shift from skills-based outcomes to reflection and discussion, by employing assessment tools 
that “enourag[e] students to collaborate and reflect on their own learning” (2015, p. 9). 

For the authors of this study, the introduction of the Framework provided an opportunity to 
reimagine the information literacy assessment program and served as the guiding document 
of a new outcomes-based assessment model. The authors present a practical application of 
the Framework to a programmatic information literacy instruction assessment, addressing 
both the need for thoughtful and evidence-based approaches for program-level assessment, 
and integration of the Framework. 

Institutional Context 

University of Maryland Libraries 

The University of Maryland (UMD), is the flagship institution of the University System of 
Maryland. It offers 91 undergraduate majors and more than 200 graduate degrees across 12 
colleges and schools. The university has a total enrollment of 36,440 (27,108 undergraduates 
and 9,332 graduates); a tenured or tenure-track faculty of 1,487 among 4,509 total faculty; 
and a staff of 5,315. 

UMD Libraries is an eight-library system and one of 16 members in the University System 
of Maryland consortium. It has 219 total staff members, 69 of which are faculty librarians. 
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The libraries have a strong history of providing instruction to the university community. In 
2015, library staff led a total of 982 instruction sessions to 19,583 students and faculty, 
which included 221 sessions for ENGL101, accounting for 4,200 students. 

Pilot Program 

In spring 2015, the researchers began the process of reimagining the first-year instruction 
program. While the majority of library instructors continued to use the standardized 
teaching outline, the researchers launched a small pilot using a revised active-learning based 
lesson plan grounded in ACRL’s Framework for Information Literacy. To assess the 
effectiveness of the new lesson plan, the researchers created a simple assessment: at the end 
of each pilot session, students would be asked to share their “a-ha moment”—a moment 
from the learning experience that stood out to them, or had personal meaning—in a single 
sentence. Based on the concept of the six-word memoir (Miller, 2011), the “a-ha moment” is 
intended to honor individual experience, while encouraging learners to synthesize their 
thoughts into a short, discrete statement that could be read and evaluated by instructors. To 
mitigate the time needed to explain the assessment, students were asked to tweet their “a-ha 
moment” using the hashtag #mylibrarymoment. Rather than measuring the ability to 
accomplish a specific set of skills, the “a-ha moment” was designed to capture the multiple 
realities that exist in any one teaching experience and value individual voices. 

Methods 

The “a-ha moment” assessment was piloted in 12 one-shot instruction sessions for first year 
composition led by two full-time library instructors and taking place between February and 
March of 2015. Over the course of the 12 sessions, 142 responses were collected, for a 
response rate of 62%. “A-ha moment” responses were collected from Twitter using two 
web-based programs: “If This Then That” (IFTTT) and “TAGS.” Each of these required a 
recipe, or a specific set of circumstances that, when fulfilled, prompted the systems to 
identify, collect, and archive the tweet. Tweets including #mylibrarymoment were 
automatically copied and saved to a Google Sheet. 

Early in the pilot, the researchers identified a challenge in using Twitter as a collection tool; 
not only had we overestimated the number of students who were active on Twitter, but also 
the amount of information students would be comfortable disclosing in a publicly-accessible 
space. An informal survey revealed that anywhere from one-third to one-half of students 
were active on Twitter, and of those, approximately half had private accounts. This meant 
that even when a student authored and published a tweet using #mylibrarymoment, if the 
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account s/he tweeted from was private, the collection tools were prohibited from accessing 
or archiving the tweets. 


While Twitter offered the benefit of being able to respond to and share tweets publicly, the 
privacy settings made it difficult to implement this method on a large scale. As a 
workaround, the researchers added a Google form for students who did not have Twitter 


accounts or who had private ones. The form was connected to the same Google sheet used 
by IFTTT and TAGS to archive tweets using #mylibrarymoment. All responses, regardless 
of whether they were collected through the Google form or Twitter, were capped at 140 
characters. 

Although individual “a-ha moments” offered insight into the instruction experience, a 
systematic analysis method was necessary to identify trends in the data. In response, the 
researchers developed a process for organizing and coding responses based on the ACRL 
Framework for Information Literacy. Because the “a-ha moment” was intended to measure 
attitudes, rather than skills, the researchers focused on the Dispositions outlined in the 
Framework. As a team, the researchers identified seven Dispositions, pulled from five of the 
six threshold concepts that best aligned with the learning outcomes for the course and 
represented an appropriate developmental level for first year learners. Each “a-ha moment” 
was read by the team of researchers and assigned to the Disposition that best matched the 
content. The analysis was based on consensus, with researchers discussing each response 
and its appropriate placement, deliberating until a unified decision had been reached. Table 
1 provides an overview of the Dispositions selected, types of responses assigned to each 
Disposition, and percentage of responses. 

Results 

While the pilot lesson plan was based on the Framework for Information Literacy and included 
discussions on a student’s role in the scholarly conversation and the construction and 
contextualization of authority, results from the pilot indicate these concepts failed to 
resonate with students, or, at the least, they did not stand out as the most meaningful 
learning experience. The recognition that “scholarly conversations take place in different 
venues,” in particular, scored very low, representing 0% of overall responses. Although the 
pilot lesson plan intentionally decreased the amount of in-class time spent on database 
demonstrations and search skills, such as Boolean operators, almost 70% (n=99) of responses 
continue to align with the Dispositions related to search and retrieval. 
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Table 1: Pilot Program Fall 2015 


Frame 

Example 

% N 

Authority is 

constructed 

develop awareness of the importance of 
assessing content with a skeptical stance and 

with self-awareness of their own biases and 

worldview 

“Make sure a publisher is unbiased. 
#mylibrarymoment.” 

5% 

7 

Information 

creation 

respect the original ideas of others 

“Some books have more than one 

author, so you can actually cite the 
different chapters as different sources 
#mylibrarymoment” 

6% 

9 

Information 

has value 

value intellectual curiosity in developing 
questions and learning new investigative 

methods 

“Learning how to use Research Port was 
really helpful because I was able to find 
so many more articles that will help me 
with my research.” 

49% 

70 

Research as 

inquiry 

recognize that scholarly conversations take 
place in various venues 

“Working with my peers to find 
connections between our extremely 
different topics and sharing databases 
that wouldn’t at first seem to be 
[applicable].” 

9% 

13 

Scholarship 

as 

conversation 

see themselves as contributors to the 

scholarship rather than only consumers 

n/a 

0% 

0 

Searching as 

exploration 

seek guidance from experts, such as 
librarians, researchers, and professionals 

“The librarians helped me get really 
helpful information from Research Port” 

6% 

9 

Searching as 

exploration 

understand that first attempts at searching 
do not always produce adequate results 

“#mylibrarymoment was that you could 
use synonyms to broaden your search 
within the same topic. I usually use the 

same words when I search.” 

21% 

29 

Other 


“When I found a correlation between 

contracting celiac diseases and 
consumption of Genetically Modified 
Foods” 

4% 

5 


Discussion 

The pilot was the first step in transitioning away from the quantitative and skills-based 
assessment associated with the prior instruction program. Responses from students 
identified a need for an increased attention to higher-level critical thinking skills, such as the 
evaluation of information or recognition of self-bias. The high number of students who 
continued to connect with the search-and-retrieval skills as their “a-ha moment” over 
critical thinking skills, such as authority or self-bias, indicated that while the researchers had 
made progress with the lesson plan, there was still work to be done. 
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The benefit of the “a-ha moment” was that it took very little time to explain during an 
instruction session; the entire process from introduction to collection could be completed in 
less than five minutes. However, what the assessment gained in in-class convenience, it 
suffered in the increased time needed for analysis. Development of a coding system 
(including analyzing the Framework and identifying target Dispositions), compilation of the 
data, and the requisite evaluation of each response required several hours of staff time. 
However, while the pilot assessment did increase the overall time spent on assessment, 
especially when compared to the previous four-question multiple choice survey, the 
character limit imposed by the tweet kept the process manageable. Responses could be read 
quickly and organized into the appropriate category almost immediately. Coding responses 
based on the seven Dispositions also helped create meaning from what could have been a 
disparate pool of data, enabling researchers to develop a better understanding of how 
students’ experiences connected to the student learning outcomes for the session, lesson 
plan, and Framework. 

Because of the open-ended nature of the assessment, the researchers had also initially had 
concerns about the number of irrelevant responses we might receive. However, we were 
pleased to discover that of the 142 total responses, only 5 (4%) fell outside of the parameters 
indicated in Table 1. 

Implications 

To scale up the pilot to meet the needs of the full program, the researchers revised their 
lesson plan to emphasize the student’s role in the research process, developed a more 
sophisticated rubric to evaluate and code “a-ha moment” responses, and modified the data 
collection process. However, implementation of the new program was contingent upon a 
restructuring of our training for library instructors. Extending as far back as 1999, UMD 
Libraries has hired MLIS students from UMD’s iSchool to serve as “special lecturers” to 
meet the demand for ENGL101 library instruction. Before an active and critically-based 
lesson plan could be implemented, it was important we equipped our new teachers to 
succeed by offering an increased level of training and support. 

In response, the researchers created a three semester Research and Teaching Fellowship 
(RTF). Intended to foster an intentional community of practice, the RTF has transformed 
the “special lecturer” position from an institutional crutch to a virtuous system that gives 
back to the university and the profession by providing thoughtful training and education to 
MLIS students. Rather than hiring part time instructors in the fall, as had been the previous 
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practice, Fellows begin their program in the spring when the amount of library instruction 
is less demanding. While “special lecturers” were contracted from semester to semester, 
Fellows commit to the program for a full three semesters, beginning with their second 
semester in the MLIS program and concluding with their fourth and final semester. 1 In May 
2015, the Libraries hired the first cohort of Fellows, compressing the first semester of 
reading, discussion, co-teaching, and observation into a 10-week summer program. 

Full-Scale Implementation 

In fall 2015, the researchers expanded the pilot to include the entire first year composition 
program. In addition to updating the lesson plans and implementing the Research and 
Teaching Fellowship, the researchers also revised the assessment by switching the collection 
method from Twitter to Qualtrics: a proprietary web-based survey tool. Although Twitter 
offered the benefit of being able to share and respond to tweets publicly, it created 
challenges for data collection. Instead of tweets, responses are collected through a single 
question Qualtrics survey, linked directly from the library website. The new format also 
included customized fields for students to identify their library instructor, which has 
become an important part of the evaluation process. To keep the spirit of the “a-ha 
moment,” responses continue to be capped at 150 characters. The result is a short, quick 
assessment that provides rich data at the program and instructor levels. 

Methods 

In addition to improving the collection method, the researchers also made three updates to 
the analysis process. First, we revised the rubric to include six Dispositions and one 
Knowledge practice, to address all six threshold concepts in the ACRL Framework. Second, 
we developed a more robust coding process by creating a rubric to analyze responses, which 
included criteria for “developing,” “proficient,” and “advanced” levels of competency. Finally, 
we added two additional categories: “other,” to account for responses that did not fit within 
one of the seven categories, and “comfort level with UMD library website, physical spaces or 
library instructor” (Appendix A). 

Although rubrics typically require evaluators to score a single learning object using each of 
the categories represented, the “a-ha moment” rubric asks evaluators to associate each “a-ha 
moment” with a single category, and then to assign a developmental level (developing, 
proficient, or advanced) within that category. The only exception would be a response 
which is indicative of both a “comfort level with UMD library” as well as a concept, such as 


Gammons & Inge 

Using the ACRL Framework... 


[ Article ] 



Communications in Information Literacy | Vol. 11, No. 1,2017 



the ability to “match an information need with an appropriate resource/’ in which case the 
response would be categorized in both areas. 

A final change from the pilot to the full-implementation of the program was an increase in 
number of evaluators. While the pilot was limited to two full-time librarians, who worked 
in concert to design, implement, and evaluate the assessment, the full-program called for 
participation by nine staff members (two librarians, two graduate assistants, and five 
Fellows), who were responsible for teaching ENGL101 library instruction sessions and also 
participated in the evaluation and analysis of “a-ha moments.” 

To norm the rubric, the researchers pulled a random sample of 50 from the more than 1,300 
total responses collected during fall 2015. As a group, the evaluators discussed each response 
at length, coming to a consensus on the category and level of competency best represented. 
Based on this discussion, the researchers made slight modifications to the levels of 
competency. This process continued in spring 2016, with two changes. In fall 2015, the first 
semester the assessment was implemented to scale, the researchers built in an extra level of 
scrutiny by having each response evaluated by two evaluators. The expectation was that the 
norming process would standardize the evaluation and that each evaluator would assign the 
response to the same category and level of competency. However, compilation of the final 
rankings revealed inconsistencies resulting from evaluators who had interpreted responses 
to be indicative of different levels of competency, or less often, different Dispositions or 
Knowledge Practices. These issues were resolved in spring 2016 by increasing the amount of 
responses put through the norming process from 50 to 100, as well as the creation of a list of 
example responses for each criteria and level of competency (Appendix B). 

After the norming, analysis proceeds as follows: 

1. At the end of the semester, each library instructor is provided with a spreadsheet 
with his/her student responses, pulled from the Qualtrics form. 

2. Instructors evaluate their set of responses, assigning each to the appropriate 
category and level of competency. 

3. Researchers collect the final spreadsheets and use the data to compile a report both 
at the macro-level and by individual instructor. Reports show the distribution of 
responses across the categories and levels of competency. 


[ Article ] 


Gammons & Inge 
Using the ACRL Framework... 



178 


Communications in Information Literacy | Vol. 11, No. 1,2017 


Results 

Results from fall 2015 and spring 2016, as shown in Table 2, indicate that students found 
their most significant areas of learning to be “information creation as process” (59%, 
n=1948), which corresponds with the ability to match an information need with an 
appropriate library resource. The second most significant area, “searching as strategic 
exploration” (23%, n=783), relates to search strategies and the ability to seek guidance from 
experts. Few results were indicative of the higher level critical thinking skills, such as 
“authority is constructed and contextual” (7%, n=222) or “information has value” (2%, n=82). 

Within the six categories, a majority of students performed at a “developing” level (60%, 
n=2018), and about a third of students performed at a “proficient” level (33%, n=l 100). Few 
students demonstrated an “advanced” level of competency in any of the categories (7%, 
n=248). While Dispositions associated with higher level thinking skills, such as 
“information has value,” were less often represented, students that did share responses 
indicative of those concepts tended to connect with those ideas more deeply (ex: 63% 
students performed at an “advanced” level when sharing responses related to “information 
has value”). 

Table 2: Responses from Fall 2015 through Spring 2016 coded to the appropriate ACRL Frame and 
developmental level 


ACRL Frame 

Developing (1) 

Proficient (2) 

Advanced (3) 




%N (n) 

%N (n) 

%N (n) 

Mean 

% (N) 

Authority is constructed 

and contextual 

75% (165) 

25% (55) 

0% (2) 

1.26 

7% 

(222) 

Information creation as 

process 

62% (1207) 

33% (651) 

5% (90) 

1.42 

59% 

(1948) 

Information has value 

25% (21) 

12% (10) 

63% (51) 

2.23 

2% (82) 

Research as inquiry 

59% (166) 

31% (86) 

31% (28) 

1.50 

8% 

(280) 

Scholarship as 

conversation 

18% (9) 

27% (14) 

55% (28) 

2.37 

1% (51) 

Searching as strategic 

exploration 

57% (450) 

37% (284) 

6% (49) 

1.58 

23% 

(783) 

Total 

60% (2018) 

33% (1100) 

7% (248) 

Total N = 

= 3366 
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Discussion 

In many ways, the results above are appropriate for a first-year student audience. It is 
reasonable and appropriate that first year students visiting the library early in their academic 


career would perform at a developing or proficient level in any of these criteria. The fact 
that some of the learners (7%, n=248) were able to demonstrate an “advanced” level of 


competency in any of the criteria is impressive. It is also important to contextualize the 
results within the broader arc of our instruction program. Had these responses been 
solicited even a year earlier, it is probable that all of them would have centered on 
“information creation as process,” or the ability to match an information need with the 
appropriate library resource; that approximately 40% of responses were indicative of other 
conceptual frameworks shows enormous growth in our instruction program. However, the 
emphasis by students on the resources and tools for research, indicates a need for instructors 
to better frame a conversation about databases and search strategies within the broader 
research process. It should also be noted that because this is a first-year instruction session 
with new library users, an orientation to the library system and its resources will always be a 
necessary component of the library lesson plan. It is reasonable to assume that some 
students will continue to connect with these ideas as a takeaway from the session, although 
we can make a more concerted effort to contextualize these processes within a broader 
discussion. 

In addition to providing an opportunity to hear directly from our learners, the “a-ha 
moment” has also allowed us to critically evaluate our work as teachers. During the 
assessment cycle, which occurs once a semester, evaluators come together as a group: first, 
to norm a random sample of responses in preparation for coding, and second, to reflect on 
the process and discuss results after the coding has been completed. While the coding takes 
place individually, reflection takes places as a community and is dedicated to improving 
practice both at the individual and program levels. During the group discussion, evaluators 
compare reports generated at the micro (individual) and macro (program) levels. If a 
particular instructor—as compared to the overall responses—has a more challenging 
Disposition, such as “seeks out conversations taking place in their research area” represented 
more often, or her responses demonstrate a comparatively advanced level of competency, 
the instructors will discuss the individual's approach to instruction. These discussions about 
individual praxis offer opportunity for rich dialogue around the teaching and learning 
process. The granularity of this approach gives insight not only into the work of the large- 
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scale program, but also individual growth and process, which is particularly important for 
Fellows who are developing their teaching skills. 

Limitations 

The most significant change from the pilot and the full-implementation of the “a-ha 
moment” has been the introduction of the rubric with levels of competency. Mapping the 
responses onto the rubric has enabled evaluators to identify program-wide trends in the 
data that would have been difficult to spot on an individual level. However, it should be 
noted that while the assessment includes a period of norming each semester, assigning of 
responses to categories and developmental levels continues to be subjective; responses may 
be impacted by how the library instructor introduced the assessment, when the library 
session occurred during the semester, and how the evaluator interpreted and ranked the 
results. The results provide a general, rather than specific, overview of what students found 
the most meaningful from the library sessions. This is not to say that the results are not 
valuable, but that they should be contextualized within the inherent limitations of 
qualitative research. 

Conclusion 

Integrating an iterative student-centered assessment has not only changed the approaches to 
teaching within UMD Libraries, but also provided a holistic evaluation of information 
literacy instruction at the individual and program levels. In response, the researchers have 
made radical changes to the teaching outline for first year composition, emphasizing active 
learning and discussion and minimizing lecturing and database demonstrations. The hiring 
and training of “special lecturers” has also been transformed. Rather than hiring lecturers a 
few weeks before the start of the semester, the Research and Teaching Fellowship scaffolds 
the development of teacher-training over three semesters, improving the experience of the 
MLIS student participants and increasing the overall instruction experience for first year 
students. 

As a result of these changes, the relationship between the libraries and the first-year 
composition program has improved. ENGL101 students are retaining more information 
from the session and are engaging with material on a deeper level. Library instructors are 
more satisfied with their teaching experience, which creates opportunities for more positive 
interactions with students and course instructors. Using the ACRL Framework as a guiding 
document for assessment has prompted instructors throughout the Libraries to engage 
more deeply with the Framework, incorporating it into their individual teaching practices. 
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The teaching and assessment process has also created a more student-centered and 
empathetic community of practice among instructors, who are working together to evaluate 
and improve instruction. Finally, the “a-ha moment” offers an opportunity to value our 
learners as individuals with unique and important experiences. While large-scale assessment 
can have a tendency to feel clinical and disengaging, we are proud to have created a program 
that prioritizes relationship building between library teachers and students, and allows 
students and library instructors to bring their full-selves into the assessment process. 
Moving forward, it is the hope of the researchers that more programs will implement, 
assess, and share thoughtful approaches to large-scale program-level instruction. 

Note 

1. For more information on the Research and Teaching Fellowship, see: Gammons, R., 
Carroll, A., & Inge, L. (2017). Sharing our success: Using a teacher training program 
to improve information literacy instruction and support MLIS students. Proceedings 
from ACRL 2017 available at http://hdl.handle.net/l903/l9171 
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Appendix A: A-Ha Moment Rubric 


UMD Libraries Information Literacy Dispositions Rubric 

| Developing (1) | Proficient (2) | Advanced (3) 

Authority is Constructed and Contextual 

Understands importance of evaluating 

information and demonstrates self- 

awareness of individual biases 

Writer acknowledges 

information evaluation as 

concept 

Writer articulates the rationale or 

importance of evaluating the 
credibility of a source 

Writer demonstrates the value of 

evaluating a source, and indicates an 
understanding of the role of self-bias 
in the process 

Information Creation as Process 

Matches an information need with an 

appropriate resource 

Writer acknowledges 
that different resources 

are available for research 

Writer identifies a type, purpose, 
or title of a specific resource 

Writer articulates how a specific 
resource addresses their individual 

information need 

Information has Value 

Respects the original ideas of others 

Writer acknowledges 

attribution methods 

Writer articulates the value of 

attribution 

Writer articulates the importance of 
attribution and identifies resources for 
help/attribution methods. 

Research as Inquiry 

Values intellectual curiosity in 

developing questions; Consider 

research as open ended exploration 

and engagement with information 

Writer acknowledges 
research as concept 

Writer acknowledges research as 

process 

Writer articulates the iterative process 
of developing / defining a research 
question. 

Scholarship as Conversation 

Seeks out conversations taking place in 

their research area 

Writer acknowledges 
that there are different 
points of view on a topic 

Writer articulates the need to 
incorporate different points of 
view 

Writer demonstrates the value of 
incorporating different points of view 

Searching as Strategic Exploration 

Designs and refines search strategies as 

necessary 

Writer acknowledges 
search strategies for 
narrowing or broadening 

Writer articulates specific search 
strategies (such as key terms, 
subject thesaurus, etc...) 

Writer demonstrates awareness of 
search strategies and how they can aid 

in student research 

Seeks guidance from experts such as 

librarians, researchers, and 

professionals 

Writer acknowledges 
assistance available 

Writer acknowledges assistance 
available and identifies ways to 
get in contact with appropriate 
professionals 

Writer articulates specific ways 
appropriate professionals can support 

students 

Other 


Comfort with UMD library website, 
physical spaces, or library instructor 


Other 
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Appendix B: Examples of student responses coded to the appropriate ACRL 
disposition/knowledge practice and developmental level 


UMD Libraries Information Literacy Dispositions Rubric 


Developing (1) 

Proficient (2) 

Advanced (3) j 

Authority is Constructed and Contextual 

Understands importance of 

evaluating information and 

demonstrates self-awareness of 

individual biases 

“I learned what a scholarly 
source is!” 

“That not all articles are 
trustworthy, always make sure to 
see the legitimacy of the articles 
you decided to use as a source” 

“My a-ha moment was when we 
researched the credibility of George 
Zornick. I realized the importance of 
studying the author's history.” 

Information Creation as Process 

Matches an information need with 

an appropriate resource 

“I realized that I should be using 
the library's resources instead of 
normal search engines to yield 
more refined results.” 

“My a-ha moment was the CQ 
researcher database. It’s really 
cool and seems helpful for 
preliminary information” 

“I was impressed by CQ researcher, 
which gives a great overview of my 
research topic and which helped me 
think of new subtopics to research.” 

Information has Value 

Respects the original ideas of 

others 

“Today I learned a lot about 
how to fmd valid sources and 

how to cite them. This will be 
very helpful when writing my 
paper." 

“I was able to effectively learn 
how to properly cite in MLA 
with clear instructions from the 

instructor.” 

“EBSCO citations are always 
capitalized and I did not know that 
they were incorrect. Now I know to 
double check so people can find my 

sources.” 

Research as Inquiry 

Values intellectual curiosity in 

developing questions; Consider 

research as open ended exploration 

and engagement with information 

“I have a variety of resources 
online and in-person that I can 
come to! Doing research can be 
easy” 

“Today's lesson helped me with 
structuring the way I do my 
research. I know how to start a 

broader search and then refine 

it.” 

“The stasis theory can allow me to 
find out what I already know about 
my topic and what question I can 
ask to make my researching process 

easier!” 

Scholarship as Conversation 

Seeks out conversations taking 

place in their research area 

“LOVE the pro/ con link on the 
cqpress! Especially for just 
starting research.” 

“Learning about CQ Researcher 
and how you can find great 
balanced information on 

controversial issues” 

“Finding possible research topics on 
CQ, and reading through the 
Pros/Cons section to get a better 
sense of the argument!” 

Searching as Strategic Exploration 

Designs and refines search 

strategies as necessary 

“Learning to put filters on my 
sources. I'm surprised no one 

has ever told me to do that.” 

“I learned that you can narrow 
the search on Academic Search 

Complete by using the thesaurus 
terms to find other key terms.” 

“On EBSCO I did not know that 

other websites were available to find 

specific articles that fall under 
education or psychology.” 

Seeks guidance from experts such 

as librarians, researchers, and 

professionals 

“A library instructor can help if 
a link doesn't work” 

“The library has the help chatting 
page which is extremely useful” 

“I like the "I have a question" form. 

It will be useful when I need to ask a 
question about my assignment or 
need help with it.” 

Other 

Comfort with UMD library 

“OMG McKeldin has EVERYTHING <3 / This was extremely informative, and Rachel was super nice and I 

website, physical spaces, or library 

helpful! 



instructor 




Other 
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